Conventional computer memory subsystems are often implemented using memory modules. A computer circuit board is assembled with a processor having an integrated memory controller, or coupled to a separate memory controller. The processor having the integrated memory controller or the separate memory controller is connected by a memory bus to one or more memory module electrical connectors (the bus may also connect to additional memory permanently mounted on the circuit board). System memory is configured according to the number of and storage capacity of the memory modules inserted in the electrical connectors.
As processor speeds have increased, memory bus speeds have been pressured to the point that the multi-point (often referred to as “multi-drop”) memory bus model no longer remains viable. Referring to FIG. 1, one current solution uses a “point-to-point” memory bus model employing buffered memory modules. In FIG. 1, a computer system 100 comprises a host processor 105 communicating across a front-side bus 108 with a memory controller 110 that couples the host processor to various peripherals (not shown except for system memory). Memory controller 110 communicates with a first buffered memory module 0 across a high-speed point-to-point bus 112. A second buffered memory module 1, when included in system 100, shares a second high-speed point-to-point bus 122 with first memory module 0. Additional high-speed point-to-point buses and buffered memory modules can be chained behind memory module 1 to further increase the system memory capacity.
Buffered memory module 0 is typical of the memory modules. A memory module buffer (MMB) 146 connects module 0 to a host-side memory channel 112 and a downstream memory channel 122. A plurality of memory devices (Dynamic Random Access Memory Devices, or “DRAMs” like DRAM 144, are shown) connect to memory module buffer 146 through a memory device bus (not shown in FIG. 1) to provide addressable read/write memory for system 100.
As an exemplary memory transfer, consider a case in which processor 105 needs to access a memory address corresponding to physical memory located on memory module 1. A memory request issues to memory controller 110, which then sends a memory command, addressed to memory module 1, out on host memory channel 112. Memory controller 110 also designates an entry 115 corresponding to the memory command into replay queue 111. Prior entries corresponding to prior memory commands may be ahead of entry 115 in queue 111.
For tractability reasons, entry 115 may be retired from the queue 111 only after two conditions are met. First, memory controller 110 only retires an entry after a corresponding non-error response is received. Second, memory controller 110 only retires an entry if all prior entries have been retired.
The MMB 146 of buffered memory module 0 receives the command, resynchronizes it, if necessary, and resends it on memory channel 122 to the MMB 148 of buffered memory module 1. MMB 146 detects that the command is directed to it, decodes it, and transmits a DRAM command and signaling to the DRAMs controlled by that buffer. If the memory transfer was successful, MMB 148 sends a non-error response through memory module 0 back to memory controller 110. Memory controller 110 retires entry 115 from replay queue 111 after the non-error response is received, but only if all prior entries have also been retired.
Due to economies, the size of the replay queue 111 is limited. Therefore, entries need to be retired as quickly as possible. Due to northbound bandwidth limitations of high-speed point-to-point bus 112, receipt of non-error responses such as write acknowledges may be delayed. Delayed receipt of such a write acknowledgement may in turn delay the retirement of subsequent entries that were entered into replay queue 111 after entry 115. The delayed retirement of an entry and subsequent entries limits the amount of space available in replay queue 111 for new entries.
Because of the forgoing limitations, the amount of free space in replay queues of memory controllers is limited. The disclosure that follows solves this and other problems.